Marked-Up Version of Substitute Specification^-^ 

D e scription ^ 

PRODUCTION OF A COMPLETE IMAGE BY SCANNING PARTIAL 
AREAS OF A PATTERN 

5 BACKGROUND 

The inv e ntion present disclosure relates to a method according to the 
pr e ambl e of Claim l for producing a resulting image from a plurality of individual 
images . 

Such a method can, for example, be used to obtain a panoramic image^ or 

10 alse-to scan documents. 

The solid angle given by a camera and objective^ which that is scanned or 
recorded by an image a is in many situations inadequate for completely recording 
objects from the surroundings. The use of_ Iin particular^ the use of wide-angle 
objectives to completely record objects is not always possible^ or always desired. 

15 Because of the characteristics of the-lenses, a very wide-angle objective causes 
extreme distortions at the edges of the image whieh -that interfere considerably with 
the impression. _Furthermore 3 the quality of the display of the object display is 
disturbed by the limited sensor resolution. In addition, in many cases, how e ver, 
replacement of the lens or of the objective is not possible-either, because this-the 

20 component is firmly integrated into the camera. 

Many previous methods hav e alr e ady b e en used which have attempted to 
dealt with a similar subject _. Th e focus of th e obs e rvation h e r e was fr e qu e ntly 
e nrelated to the generation of panoramic images^ Te-correcting the sensor defect? 
Hhowever, the known previous methods for generating images of a larger field of 

25 vision require very expensive and complex arithmetical operations that cannot be 
performed in real time video tim e to correct the lens and image distortion. For this 
reason, a calculation of an image with an extended field of vision can generally 
only be performed offline as a rule . 

The images used to generate the resulting image of the previous methods 

30 only require a slight overlap in this mothod in the individual images . In this -those 
cases, two adjacent images or two consecutive images contain only a little common 



614180/D/l 



9 



image information. Furthermore, ^ depending on the environment , how e v e r , the 
images ther e for e — int e grally may contain quite different lighting conditions. 
Therefore, as a result of Aan aperture control on the camera, how e v e r, r e sults 
ffl there will exist extremely varied illuminations at the spliced edges of the resulting 
5 image,, whieh -that can likewise only be harmonized again at a^very great cost. 

Fax equipment and flatbed scanners enable the electronic transmission of 
documents which are available on paper. In many situations th e availability of this 
type of equipment is not available , for example a during a meeting , is not always 
guarant ee d . An alternative to the use of a fax machine or a flatbed scanner is such a 

10 situation is a digital camera, with which the image information can alse-be saved in 
digital form. However, the resolution of standard commercial cameras is not yet 
sufficient to produce adequate resolution for a document in a single exposure. 

The scanning of documents by means of many individual images close up to 
the document to increase the resolution is a known approach. Until now , however, 

15 there have only been a few, in part, very unstable methods that can be used to 
compose a two-dimensional complete image from the individually recorded images 
again . 

Most of these methods are based on the image information a^first being 
distorted at an initial processing stage by highly complex arithmetical operations. 

20 Then, a At a subsequent processing stage, the adjacent image information is then 
harmonized with the adjacent images in all four edge directions. Each image is 
corrected both horizontally and vertically in relation to the adjacent images. The 
images^ therefore, only require a slight overlap fef-to accomplish this. _Eaoh imag e 
is corr e ct e d both horizontally and v e rtically in r e lation to th e adjac e nt imag e s. 

25 Besides the scarcely tolerable high calculation time, these methods frequently lead 
to rhombic distortion of the image information which distracts the observer greatly^ 
as shown, for example, in -fefe Figure 1). _On account of their complexity, the 
known methods can frequ e ntly generally only be performed interactively^ and 
offline. 

30 SUMMARY 



614180/D/l 



The obj e ct of the invontion present disclosure is- relates to indicat e a method 
of the typ e mentioned at the beginning whioh for providinges a high-quality 
resulting image from a plurality of individual images, without a great amount of 
calculation. 

5 

This obj e ct is achi e v e d An example of the method for producing a resulting 
image from a plurality of individual images according to the invention present 
disclosure comprises producing the plurality of individual images using a scanning 
movement, offsetting the individual images in relation to each other, obtaining 
10 image information from the individual images as a function of the offsetting, and 
producing the resulting image using the obtained image information. by th e f e atur e s 
indicat e d in Claim 1 . 



Some benefits of T the method according to the invention present disclosure 
15 include has th e following advantag e s: 

the Rrapid processin g of the images to produce a resulting image, 
elimination of the need to C correctien ef-the lens errors or mapping errors 
in the image involving utilizing intensive calculations is not n e c e ssary ^ 

elimination of the need to H harmonizeatien e£ -the illumination at the 
20 spliced edges is r e nd e r e d unn e c e ssary by virtually continuous imag e scannin g , and 
c 

Control ling ef-the additional image information as a function of the imag e 
misalignm e n t ffsetting . 

Th e inv e ntion is d e scrib e d in mor e d e tail h e r e inafter with r e fer e nc e to an 
25 e x e mplary e mbodim e nt shown in th e diagram. Th e diagram shows: 
BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows an example of a compl e t e resulting image prepared 

from individual exposures according to the prior art ? . 

Figure 2 shows an example of a result ing image generated by 

30 scanning the surroundings^ 



614180/D/l 



Figure 3 illustrates an example of _a scanning movement for scanning a 

document^ 

Figure 4 illustrates an example of — the image recorded during the 
scanning movement according to Figure 2 ?i 
5 Figure 5 illustrates an example of the result of successive one- 

dimensional scans , and .. 

Figure 6 illustrates an example of the resulting image according to the 

m e thod according to the inv e ntion present disclosure . 

DETAILED DESCRIPTION OF THE PRESENT EXAMPLES 
10 The method according to the inv e ntion present disclosure is based on the 

production of individual images which are generated during a one- or two- 
dimensional scanning movement. 

According to the inv e ntion present disclosure individual images are 
generated which overlap each other to a high degree. This produces a compl e te 
15 resulting image with almost no distortions which that covers a very large solid 
angle. The mothod This also makes it possible, for example, to produce panoramic 
images or alse-to scan in documents with a very high resolution. 

In doing so, the image sensor operates, for example, at full scanning 
frequency^ so that by slowly swiveling across the object to be recorded, adjacent 
20 images are only slightly offset against each other. From each image a a part of the 
undistorted image information is copied from the center of the image into a 
resulting image. The size of the copied picture portion detail is controlled as a 
function of the calculated offset to the previous image. As two adjacent images 
overlap to a very high degree, the images have almost identical lighting conditions 
25 so that when adjacent edges are put together, as a rule no harmonization of the 
illumination is necessary. 

To p e rfoi mAn embodiment of the method according to the inv e ntion p resent 
disclosure could , for example, be used with a mobile telephone having a with the 
following compon e nts is us e d: 
30 Gcamera module on th e r e v e rs e of th e mobil e t e l e phon e for recording 

images from the surroundings* 



614180/D/l 



Pa processor for processing the-image data and for generating the result ing 
image (i.e., estimate of movement, image composition, etc.) , and a 
©display for displaying the resulting image, display 

Th e function to An example of g eneratinge the resulting image (cf. also as 
5 shown in, for example. Figure 2 A ) on the basis of an image sequence A essentially 
comprisos includes the following processing stages: the 

fiestimatione of movement of two adjacent images , as well as the 
Sstructure of the resulting image,, 

In th e following s e ctions the individual proc e ssing stag es of t The above 

10 methods are explained in more detail herein with the simplification assumption, for 
illustrative purposes only, that the scan follows the surroundings one- 
dimensionally from right to left^ 

Regarding the 4 4Eestimatione of movement of two adjacent imageSi 

Tthe movement of two adjac e nt imag e s is determined, for example, 

15 according to the-a^MSE (Mean Squared Error) method. In th e MSE i s-method, the 
best possible match in a local neighborhood to the previous image is sought for an 
image area of the initial image. The best match provides the displacement vector of 
the two images relative to each other. 

Regarding the £M Jstructuring o fp dating the result ing image a 

20 Qusin g n the basis of the displacement vector, the position inside the 

resulting image to which the additional image information is copied is ascertained. 
The width of the additional pictur e copied portion detail of the initial image is 
provided by the offset of the images in the direction of scannin g, for this example 
in the horizontal direction from right to left ( i.e., direction x). In this way no gap 

25 arises between the pictur e copied portion detail already put together and the added 
pictur e copied portion detail. V e rtical Perpendicular to the direction of scanning 
(i.e., direction y) a the complete image information is taken into account. The result 
of the processing of many images according to this method can be seen , for 
example, in Figure 1 . 

30 In a dev e lopm e nt of tho method aA ccording to the inv e ntion present 

disclosure , scanning of the document takes place by means of a zigzag movement 
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of the camera across the document at a constant distance , as shown in, for example. 
(efe-Figure 3). _In order to avoid variations in the distance to the document and 
consequently changes in the size of the images during the scanning process, for 
example, the camera is guided across the document on a frame with the camera 
pointing vertically downwards towards the document to be scanned . _By analyzing 
the movement of consecutive images, the two-dimensional scan is separated into 
several one-dimensional (i.e., horizontal) scans as shown, for example, in fc fe 
Figures 4 and 5). _Each one-dimensional scan shows a horizontal, undistorted strip 
of the original document By separating the two-dimensional scan into several one- 
dimensional scans, the probl e m of document generation task is reduced to 
horizontal scaling of the image strips^ and the putting together of the horizontal 
strips in a vertical direction , as shown, for example, in - fefi-Figure 5)._ By fc l-inearly 
scaling ef-all of the image information^ avoids rhombic distortions are avoided in 
the complete resulting image, as occurs in traditional previous methods , as shown 
in, for example, - fefe-Figure 6). 

It should be understood that various changes and modifications to the 
presently preferred embodiments described herein will be apparent to those skilled 
in the art. Such changes and modifications can be made without departing from the 
spirit and scope of the present invention and without diminishing its intended 
advantages. It is therefore intended that such changes and modifications be covered 
by the appended claims. 
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Abstract 
ABSTRACT 

The invention p resent disclosure relates to a method for obtaining 
producing a resulting image from a plurality of individual images. The method 
5 includes using producing a plurality of individual images which can b e 
s ucc e ssively produc e d durin g using a scanning movement^ — According to th e 
inv e ntion, th e succ e ssiv e and offsetting the individual images ar e slightly offs e t in 
relation to each other i7 and — The method further includes obtaining image 
information is obtain e d from the individual images according to the misalignment 
10 offsetting, and copi e d into th e producing the resulting image using the obtained 
image information . 
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This listing of claims will replace all prior versions, and listings, of claims 
in the application: 
Listing of claims: 

Claims 1-2 (cancelled). 

Claim 3 (new): A method for producing a resulting image from a plurality 
of individual images, the method comprising: 

producing the plurality of individual images using a scanning movement; 

offsetting the individual images in relation to each other; 

obtaining image information from the individual images as a function of the 
offsetting; and 

producing the resulting image using the obtained image information. 

Claim 4 (new): The method as defined in claim 1, wherein the scanning 
movement is two-dimensional. 
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REMARKS 



The present amendment makes editorial changes and corrects typographical 
errors in the specification, which includes the Abstract, in order to conform the 
specification to the requirements of United States patent practice. No new matter is 
added thereby. The present amendment also includes a Substitute Specification 
including a marked-up version of the changes made to the specification by the 
present amendment. 

In addition, the present amendment cancels original claims 1-2 in favor of 
new claims 3-4. Claims 3-4 have been presented solely because the revisions by 
red- lining and underlining that would have been necessary in claims 1-2 in order to 
present these claims in accordance with preferred United States patent practice 
would have been too extensive and burdensome. The present amendment is 
intended for clarification purposes only and not for substantial reasons related to 
patentability pursuant to 35 U.S.C. §§101, 102, 103 or 112. Indeed, the 
cancellation of claims 1-2 does not constitute an intent on the part of the Applicants 
to surrender any of the subject matter of claims 1-2. 

Early consideration on the merits is respectfully requested. 
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